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1.  Introduction 

Scientific  discovery  follows  a  cyclic  pattern  of  theory  development  -  the  formation  of  a  new 
theory,  a  number  of  fixes  to  account  for  anomalous  behavior  and,  later,  when  the  fixes  result  in  a 
grossly  inelegant  theory,  a  paradigm  shift  that  radically  transforms  the  old  theory  (Kuhn,  1970). 
Theory  development  is  a  multi-faceted  process  involving  the  synergistic  cooperation  of  a  number 
of  approaches.  This  paper  attempts  to  provide  a  comprehensive  model  for  each  stage  of  theory 
development  by  integrating  several  techniques  -  empirical  learning,  analogical  learning,  and 
experimentation. 

Our  central  task  is  to  provide  causal  explanations  of  observed  natural  phenomena,  using  an 
imperfect  or  non-existent  theory  of  the  domain.  This  paper  describes  the  integration  of  two 
techniques:  verification-baaed  analogical  learning,  which  is  used  primarily  for  theory  formation 
(including  paradigm  shifts),  and  experimentation-baaed  theory  revision,  which  is  used  primarily 
for  theory  testing  and  revision.  (We  review  each  only  briefly  here,  for  details  see  (Falkenhainer, 
1987)  and  (Rajamoney,  1988)).  Analogy-driven  theory  formation  can  develop  novel  theories  in  a 
focused  manner,  drawing  from  its  knowledge  of  analogous  precedents.  However,  in  isolation  it 
lacks  the  ability  to  examine  the  state  of  the  world  and  experimentally  confirm  or  refute 
hypotheses.  Experimentation-based  theory  revision,  on  the  other  hand,  is  well  suited  to 
modifying  theories  and  testing  the  different  modifications  to  a  theory.  However,  it  has  very  little 
guidance  during  the  task  of  proposing  different  modifications  to  a  theory.  Also,  it  must  rely  on 
fortuitous  observations  or  prediction  failures  to  initiate  theory  revision. 

In  this  paper,  we  describe  a  set  of  general  principles  underlying  theory  formation,  theory 
revision,  and  experimentation.  We  demonstrate  how  these  techniques,  each  focusing  on  different 
aspects  of  the  problem,  may  be  integrated  to  provide  a  more  unified  and  comprehensive 
treatment  of  scientific  theory  development.  In  particular,  we  show  how  experimentation  is  used 
to  empirically  guide  and  verify  the  analogy-driven  theory  formation  stage  and  how  analogy  is 
used  to  initiate  and  focus  the  experimentation-based  theory  revision  stage.  Furthermore,  we 
outline  a  general  protocol  for  communication  between  theory  formation  and  experimentation- 
based  revision  which  solves  many  of  their  individual  problems.  Experimental  queries  using  the 
protocol  are  motivated  by  empirical  questions  and  requirements  encountered  during  hypothesis 
generation  and  evaluation.  Theoretical  queries  are  motivated  by  the  need  for  plausible  hypotheses 
during  experimentation-based  theory  revision. 

The  utility  and  generality  of  this  protocol  is  demonstrated  by  an  implementation 
integrating  two  previously  autonomous  systems:  PHINEAS,  which  performs  verification-based 
analogical  learning,  and  ADEPT,  which  performs  experimentation-based  theory  revision.  When 
confronted  with  a  novel  observation  of  evaporation,  this  integrated  architecture  and  its 
associated  protocol  enabled  PHINEAS  to  empirically  question  the  necessity  of  heat  flow  (e.g.,  from 
a  stove)  in  its  evaporation  hypothesis  derived  from  boiling.  In  turn,  the  protocol  enabled  ADEPT 
to  be  advised  of  vapor  saturation  as  a  candidate  explanation  for  why  evaporation  stopped. 

We  conclude  this  section  with  a  review  of  the  two  techniques.  An  integrated  design  and  the 
general  principles  which  emerge  is  then  described  in  section  2  and  made  concrete  with  the  use  of 
an  implemented  example.  An  additional  example  is  briefly  provided  in  section  3. 
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1.1.  Verification— Based  Analogical  Learning:  An  Overview 

Verification-based  analogical  learning  (VBAL)  (Falkenhainer,  1987)  is  an  approach  to 
theory  formation  and  revision  which  relies  on  analogical  inference  to  hypothesize  new  theories, 
and  gedanken  experiments  (i.e.,  simulation)  to  analyze  their  validity.  VBAL  is  an  iterative 

process  consisting  of  four  primary  stages: 

1.  Behavior  match.  An  anomalous  observation  triggers  a  search  for  previously  understood 
experiences  that  exhibited  analogous  behavior.  The  result  of  this  stage  is  a  candidate 
analogue  and  an  initial  set  of  correspondences  between  the  two  domains.1 

2.  Theory  generation.  The  central  objective  of  the  second  stage  is  to  produce  a  fully 
operational  initial  hypothesis  about  the  current  domain.  This  stage  has  two  components. 
First,  the  model  used  to  explain  the  recalled  experience  is  analogically  mapped  into  the 
current  domain.  This  mapping  is  guided  by  the  initial  correspondences  found  in  the 
behavioral  comparison.  Second,  the  model  must  be  operationalized  if  it  is  not  already.  F or 
example,  it  may  reference  entities  and  properties  that  are  currently  unknown.  These  entities 
and  properties  must  either  be  inferred  from  the  domain  theory  or  their  existence  must  be 
postulated. 

3.  Theory  completion.  The  operational  model  may  not  necessarily  conform  to  the  present 
observation.  Thus,  theory  revision  may  be  required  to  examine  predictions  and  produce  a 
consistent  initial  model.  Revision  in  VBAL  is  based  on  passive  consistency  -  when  empirical 
experiments  are  impossible,  prefer  minimal  revisions  with  an  analogical  precedent. 

4.  Theory  revision.  While  the  current  domain  model  may  accurately  account  for  all  known 
observations,  a  new  situation  may  be  observed  for  which  the  model  is  inadequate. 

PHINEAS,  the  current  implementation  of  VBAL,  uses  Forbus’  (1984,  1986)  Qualitative  Process 
theory  to  represent  and  reason  about  change  in  the  physical  world,  and  a  modified  version  of 
Centner’s  (1983)  Structure-Mapping  theory  for  analogical  mapping.  The  system’s  primary 
analogy  component,  SME  (Falkenhainer  et  al,  1986),  is  used  to  perform  comparisons  of  similarity 
and  map  theories  across  domains.  PHINEAS  has  been  shown  to  discover  a  caloric  theory  of  heat 
by  analogy  to  liquid  flow.  It  has  also  developed  an  explanation  of  oscillatory  electrical  circuits 
from  its  knowledge  of  mechanical  oscillation. 

1.2.  Experimentation-Based  Theory  Revision:  An  Overview 

Experimentation-based  theory  revision  (Rajamoney,  1988)  is  an  approach  to  changing  or 
augmenting  an  incomplete  or  incorrect  theory.  Experimentation-based  theory  revision  consists 
of  three  main  steps: 

l.  Contradiction  detection.  Problems  with  the  existing  theory  are  detected  by  comparing 
the  predictions  based  on  the  theory  with  the  observations  made  from  the  real  world.  If  the 
predictions  and  observations  lead  to  a  contradiction  then  the  existing  domain  theory  has  to 
be  revised. 


1  The  term  aisaiopae  is  used  in  its  most  general  sense  throughout  this  psper.  An  analogy  may  be  found  within  the  same  domain, 
as  between  one  instance  of  liquid  flow  and  another,  or  across  domains,  as  between  electron  flow  and  liquid  flow  No  strong  claims  are 
made  about  the  access  mechanism,  whose  theoretical  underpinnings  are  still  under  development.  While  implemented  and  autonomous, 
the  accester  is  far  from  sophisticated  at  this  time. 
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2.  Hypothesis  formation.  Hypotheses  that  involve  changes  to  the  existing  theory  are 
proposed.  This  stage  is  governed  by  explanation  construction  -  only  changes  leading  to  a 
revised  domain  theory  that  can  explain  the  observed  phenomenon  are  proposed. 

3.  Experiment  design.  Typically,  there  will  be  a  number  of  different  ways  to  change  a  theory 
to  explain  a  new  phenomenon.  Experiments  are  designed  to  identify  the  change  that  is 
consistent  with  the  actual  behavior  of  the  real  world  (see  Rajamoney,  1988).  Hypotheses 
whose  predictions  are  not  consistent  with  experimental  observations  are  rejected. 

ADEPT  is  a  system  that  demonstrates  experimentation-based  theory  revision.  It  starts  with  an 
operational  but  initially  incomplete  and  incorrect  theory  and  revises  it  to  conform  to 
observations  made  from  the  real  world.  ADEPT  has  been  demonstrated  on  a  number  of  examples. 
It  has  successfully  learned  effects  and  conditions  of  physical  processes  such  as  a  new  influence  of 
evaporation  on  the  temperature  of  the  evaporating  liquid  and  a  new  condition  for  dissolving 
solutes  requiring  the  concentration  of  the  solution  to  be  less  than  its  saturation  concentration. 

2.  An  Integrated  Model  of  Theory  Development 

Theory  development  is  a  dynamic  process  involving  the  formation  of  candidate  theories, 
revisions  to  account  for  anomalous  observations  or  achieve  simplicity,  and  experimentation  to 
test  hypotheses  and  collect  new  data.  These  activities  are  highly  interdependent  yet  highly 
modular.  They  may  be  performed  by  numerous  people  in  distant  locations,  possibly  across 
generations  (e.g.,  the  discourse  between  Copernicus,  Galileo  and  Kepler,  whose  lives  spanned 
nearly  200  years).  In  this  section  we  develop  this  "collection  of  specialized  experts"  view  of  theory 
development  with  respect  to  theory  formation,  revision,  and  experimentation  and  show  how  the 
expertise  of  each  is  required  to  reduce  the  search  space  throughout  the  development  cycle.  We 
first  describe  strengths  and  limitations  of  our  two  methodologies  and  their  complementary 
relationship.  We  then  define  a  general  protocol  for  interaction  between  systems  specializing  in 
theory  formation,  revision,  and  experimentation.  Next,  we  show  how  the  integrated  system 
provides  a  comprehensive  model  for  each  stage  of  theory  development. 

2.1.  Limitations  and  Strengths 

As  independent  research  efforts,  each  approach  is  designed  to  perform  theory  development 
autonomously.  VBAL  conjectures  explanations  of  the  new  behavior  based  on  its  similarity  to 
previously  encountered  phenomena.  Its  power  lies  in  its  ability  to  construct  novel  theories, 
focused  by  considering  only  theories  for  which  it  has  an  analogical  precedent.  However,  VBAL 
has  some  important  limitations.  It  is  intended  for  passive  observation  and  thus  does  not 
empirically  test  a  hypothesis.  It  makes  intermediate  assumptions  and  depends  on  simulation 
experiments  to  test  the  consistency  of  these  assumptions.  This  leads  to  questions  about  the 
validity  of  proposed  theories.  In  addition,  VBAL  must  rely  on  heuristics  and  prior  experiences  to 
guide  search,  thus  requiring  one  to  entertain  hypotheses  that  might  easily  be  empirically  refuted. 

In  turn,  experimentation-based  theory  revision  locates  known  theories  that  appear  relevant 
to  the  observation  and  modifies  them  to  inclusively  account  for  the  new  example.  Its  power  lies  in 
its  ability  to  interact  actively  with  the  world,  discriminate  among  alternative  revisions,  and 
refute  proposed  hypotheses.  However,  experimentation-based  theory  revision  has  some  important 
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limitations.  It  is  unable  to  make  large,  fundamental  modifications  to  a  theory,  or  produce  one 
where  none  existed  before.  This  limits  it  to  revising  "roughly  correct”  theories.  Furthermore, 
experimentation  is  essentially  open-ended  in  terms  of  what  tests  can  be  made  and  what  kinds  of 
experiments  can  be  designed.  For  example,  it  may  not  be  tractable  to  collect  all  possible 
hypotheses  prior  to  conducting  experiments. 

The  limitations  of  each  approach  corresponds  nicely  to  the  strengths  of  the  other. 
Verification-based  analogical  learning  provides: 

1.  Highly  focused,  theoretically  motivated  queries  about  the  state  of  the  world,  such  as 
checking  for  the  existence  of  an  anticipated  object  or  testing  theoretical  predictions. 

2.  Large  changes  in  theoretical  perspective,  develops  new  theories  and  provides  candidate 
theories  to  revise. 

3.  A  restricted  number  of  potential  hypotheses  and  a  preferential  ordering  on  hypotheses. 
Experimentation-based  theory  revision  provides: 

1.  Empirical  tests  for  the  relevance  of  certain  perceived  aspects  of  a  situation,  such  as  the 
necessity  of  ancillary  processes  normally  associated  with  an  analogue  theory. 

2.  Discrimination  among  plausible  hypotheses  or  refutation  of  a  single  hypothesis. 

3.  Experiments  to  detect  the  presence  of  an  unnoticed  or  conjectured  object  or  tests  for 
conjectured  properties  of  existing  objects. 

4.  Probes  of  the  situation  for  unknown  values  of  a  quantity  or  its  derivatives. 

2.2.  A  Communication  Protocol 

In  this  section  we  develop  a  general  protocol  for  theory  development  tasks  which  enables 
one  method  to  make  use  of  another’s  expertise  in  a  system  independent  manner.  If  the  process 
answering  a  query  lacks  the  appropriate  decision  procedure  or  is  unable  to  successfully  produce  a 
response,  a  value  of  UNKNOWN  is  always  acceptable.  The  protocol  results  in  an  extendible 
architecture,  allowing  new  techniques  to  be  added  or  existing  techniques  to  be  replaced  by 
improved  versions. 

A  crucial  question  concerns  how  one  tells  whether  or  not  some  fact  holds.  In  theory 
formation  and  revision,  we  must  assume  incomplete  knowledge.  Thus,  normal  ways  of  deriving 
facts  about  the  environment  must  be  redefined.  We  distinguish  between  queries  which  depend 
primarily  on  the  current  beliefs  of  the  system  and  queries  which  are  able  to  question  those  beliefs 
by  looking  for  alternate  sources  of  information.  Direct  queries  directly  probe  for  the  status  of  a 
condition  using  inference,  or  simple  empirical  tests  if  their  status  is  not  easily  deducible.  Indirect 
queries  look  for  the  effects  normally  associated  with  a  condition,  to  test  if  deductive  conclusions 
are  wrong  or  if  an  analogous  condition  is  in  effect. 

We  decompose  theory  development  into  eight  tasks,  which  suffice  for  a  wide  range  of 
interesting  problems: 

Present?  <object-or-var!able>  <  primary-conditions >  <  secondary-conditions  > 

This  queries  for  the  existence  of  a  partially  specified,  unknown  object.  Failure  of  any 
secondary  conditions  is  not  grounds  for  rejection,  but  may  be  used  to  reduce  the  set  of 
candidates  satisfying  primary  conditions.  For  example,  investigation  of  a  chemical  reaction 
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may  lead  one  to  suspect  and  test  for  the  presence  of  a  particular  catalyst  in  the  mixture. 
Teat— Value  <  quantity  >  <  value  > 

This  instructs  the  system  to  search  for  an  inferential  determination  of  the  quantity’s  value, 
or  experimentally  probe  the  situation  for  the  value.  The  value  may  be  left  unspecified,  or 
the  truth  of  a  particular  relative  value  may  be  sought,  as  in  (GREATER-THAH  bolling- 
taaparature)  or  DECREASIIG.  For  example,  hypothesis  generation  may  require  the  value  of 
a  quantity  not  reported  in  the  initial  observation. 

Test- Condition  <  relation  > 

This  determines  if  the  specified  relation  holds.  For  example,  investigation  of  heat-related 
theories  may  require  determining  if  one  object  is  in  thermal  contact  with  another. 

Test-Effects  <  relation  > 

This  is  an  indirect  query  which  instructs  the  experimentation  system  to  look  for  the 
presence  of  the  known  effects  of  a  physical  relation.  For  example,  if  Test-Condition  had 
determined  that  a  specified  fluid  path  was  not  configured  to  support  liquid  flow,  the  validity 
of  this  conclusion  may  be  questioned  by  looking  for  the  effects  of  liquid  flow  through  the 
path.  Test-Effects  may  be  used  to  seek  an  answer  when  Test-Condition  returns 
unknown,  instigate  belief  revision,  or  create  a  new  concept  analogous  to  the  given  physical 
relation. 

Necessary?  <  condition  >  <  observation  > 

The  necessity  of  a  condition  thought  to  be  essential  to  achieving  an  observation  is 
questioned  with  this  query.  For  example,  a  developing  theory  on  chemical  reactions  may 
state  that  a  particular  catalyst  is  required.  This  query  would  call  for  experiments  to 
determine  if  the  catalyst  is  actually  required. 

Discriminate  <  hypotheses  >  <  observation  > 

This  asks  the  experiment  design  system  to  construct  experiments  to  discriminate  among  a 
set  of  hypotheses  that  are  thought  to  cause  the  observation.  For  example,  during  theory 
revision,  several  modifications  of  the  original  theory  may  explain  the  novel  observation.  This 
query  finds  those  revisions  that  are  consistent  with  information  gathered  from  directed 
experiments. 

Propose— Theories  < observation  > 

This  invokes  the  theory  formation  system  to  develop  theories  that  will  explain  the 
observation.  For  example,  during  theory  revision,  experiments  may  rule  out  all  the  candidate 
revisions  or  the  revision  proposer  may  not  find  any  consistent  modifications  to  the  theory.  In 
such  cases,  the  theory  revision  system  uses  this  query  to  post  a  theory  formation  task. 

Propose-Reviaions  <  observation  >  <  theory  >  <icondltion  or  :elTect  > 

This  query  asks  the  theory  formation  system,  or  a  specialist  in  revision  hypotheses,  to 
propose  revisions  to  a  theory  that  will  enable  it  to  explain  the  observation.  The  query  must 
specify  if  the  revision  should  be  to  the  theory’s  conditioning  relations  or  to  its  set  of  effects. 

2.3.  The  Three  Stages  to  Developing  a  Complete  and  Consistent  Theory 

There  are  three  primary  stages  to  theory  development  within  the  VBAL  framework: 
Theory  Generation,  Theory  Completion,  and  Theory  Revision.  This  section  examines  the 
interaction  of  the  two  systems  in  each  of  these  stages  and  uses  an  implemented  example  to 
illustrate  the  interaction.  The  example  involves  forming  and  revising  a  model  for  evaporation. 
Throughout  this  section,  we  will  assume  that  theories  are  represented  by  a  set  of  processes  and 
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individuate  (objects)  as  defined  in  Forbus’  (1984)  Qualitative  Process  theory.  Each  process 
specifies  a  set  of  participatory  individuals  (objects  and  othei  processes),  a  set  of  conditioning 
relatione  indicating  when  it  is  active,  and  a  set  of  effects  that  apply  when  it  is  active. 

In  our  implementation  of  this  integrated  architecture,  PHINEAS  initiates  the  process  of 
explaining  new  observations  and  posts  tasks  for  ADEPT  when  its  expertise  is  required.  If  an 
observation  cannot  be  explained  by  existing  theories,  PHINEAS  attempts  to  generate  an 
explanation  by  recalling  past  observations  of  similar  or  analogous  behavior.  The  knowledge  used 
to  explain  these  previous  observations  is  used  to  analogously  explain  the  current  situation.  In 
constructing  the  explanation,  PHINEAS  may  call  on  ADEPT  to  measure  the  value  of  a  quantity  or 
determine  the  existence  of  a  hypothesized  object.  In  addition,  experimentation  and  possible 
revision  of  the  initial  hypothesis  may  then  be  required  for  this  or  future  observations.  PHINEAS 
and  ADEPT  use  the  protocol  to  interactively  solve  such  problems  during  theory  formation  and 
revision. 

2.3.1.  Theory  Generation 

Theory  generation  is  the  process  of  proposing  an  initial  theory  to  explain  a  novel 
phenomenon.  There  are  two  stages  to  theory  generation  in  VBAL.  The  first  stage  examines  the 
current  behavior  and  recalls  previously  understood  similar  behavior.  Theories  used  to  explain  an 
analogue  behavior  are  then  mapped  into  the  current  domain  and  proposed  as  an  initial 
explanation  of  the  current  phenomenon. 

The  system  next  divides  the  elements  of  each  hypothesized  theory  into  primary  processes 
and  ancillary  processes.  A  primary  process  is  one  that  affects  an  observed  quantity,  such  as  the 
temperature  of  an  object  whose  temperature  was  observed.  Ancillary  processes  are  those  that 
were  used  in  explaining  the  analogue  case,  but  do  not  directly  affect  one  of  the  current 
observables.  The  necessity  of  a  primary  process  is  assumed  and  never  experimentally  questioned. 
Ancillary  processes  are  eliminated  during  theory  formation  if  they  are  found  to  be  unnecessary. 

The  second  stage  in  theory  generation  is  to  make  the  proposed  explanations  operational.  A 
model  is  non-operational  if  it  calls  for  participatory  objects  and  processes  whose  presence  was 
not  noticed  during  the  initial  observation.  If  their  existence  cannot  be  determined  (at  this  time), 
their  existence  must  be  assumed  or  the  proposal  modified  to  not  depend  on  them. 
Operationalization  first  consists  of  instantiating  each  primary  process  by  examining  its  unknown 
individuals: 

When  the  individual  is  an  uninstantiated,  ancillary  process : 

Attempt  to  instantiate  it. 

If  not  instantiate,  test  for  VECESSABT?. 

If  necessary,  reclassify  it  as  a  primary  process,  otherwise  remove  it. 

When  the  unknown  individual  is  an  entity. 

Test  for  PRESEHT? 

If  not,  postulate  its  existence  and  store  its  known  set  of  characteristics. 

Any  ancillary  process  that  is  experimentally  determined  to  be  irrelevant  is  deleted  from  the 
generated  theory.  Remaining  ancillary  processes  must  be  instantiated,  if  they  have  not  been 
already.  However,  if  that  instantiation  calls  for  postulating  the  existence  of  an  unknown  entity, 
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(l«c«iiu’j?  <anclllary-process>  <obs«rvatlon>)  will  be  invoked  first.  If  a  later 
contradiction  occurs,  the  ancillary  processes  should  be  the  first  to  be  questioned  during  revision. 

The  final  step  in  producing  a  fully  operational  theory  is  to  test  the  analogically  proposed 
conditioning  relations  for  validity  under  the  current  conditions.  This  is  done  using  Test-Value 
and  Test-Condition.  Conditioning  relations  not  known  to  be  true  in  the  current  context  are 
temporarily  removed  in  order  to  obtain  a  fully  operational  theory  and  enable  discrimination  with 
competing  theories.  For  surviving  theories,  an  attempt  is  then  made  to  replace  these 
conditioning  relations  with  analogous  relations  applicable  to  the  current  context. 

2.3.I.I.  Evaporation  Example:  Generating  Initial  Hypotheses 

This  section  describes  how  the  integrated  implementation  generates  an  initial  model  for 
evaporation.  The  system  begins  with  knowledge  of  six  processes  -  liquid  flow,  heat  flow, 
dissolving,  spring  oscillation,  boiling  and  heat  replenishment  (to  constantly  maintain  the  heat  of 
a  stove).  It  also  has  a  database  of  physical  observations  fully  explained  by  these  processes. 
Suppose  an  open  beaker  containing  alcohol  is  left  on  a  table  top  for  a  day,  during  which  time  the 
amount  of  alcohol  is  observed  to  decrease.  The  problem  is  to  propose  an  explanation  for  this 
observation,  which,  according  to  the  system’s  existing  knowledge,  should  never  have  happened. 

Model  Creation:  At  this  point,  no  known,  instantiate  theory  is  able  to  even  partially 
explain  the  loss  of  alcohol  in  the  beaker.  By  examining  its  knowledge  of  “similar”  behaviors, 
PHINEAS  determines  that  the  only  known  examples  of  a  liquid  leaving  a  container  call  for  it  to 
flow  out  or  to  boil  away.  Three  ordered,  initial  hypotheses,  based  on  overall  behavioral 
similarity,  are  generated.  The  first  calls  for  the  alcohol  to  flow  out,  the  second  calls  for  it  to 
vaporise,  and  a  distant  third  calls  for  it  to  dissolve  based  on  its  gradual  disappearance.  The 
initial  vaporization  explanation  is  shown  in  Figure  I.  It  consists  of  one  primary  process  -  boiling, 
and  two  ancillary  processes  -  heat  flow  and  heat  replenishment,  which  are  believed  to  be  required 
for  boiling  to  take  place. 

Model  Operationalization:  The  three  models  Abased  on  flow,  boiling,  and  dissolving)  call  for 
the  presence  of  objects  that  were  undetected  in  the  initial  observation.  We  focus  first  on 
operationalizing  the  vaporisation  explanation.  This  explanation  requires  the  presence  of  two 
unknowns  -  alcohol  vapor  in  the  beaker  and  a  heat  source.  PHINEAS  first  produces  the  query: 

(Present?  ? STEAM  1  ( (Contalned-Gas  ?STEAM1)  primary  condition t 

(Container -of  ?STEAM1  beaker) 

(Substance-of  7STEAM1  alcohol)) 

())  ;no  secondary  conditions 

Notice  that  the  query  is  very  specific  -  the  object  is  fully  specified  and  all  that  remains  is  to  test 
if  it  is  there.  Of  course,  even  this  test  can  be  non-trivial.  To  answer  the  query,  ADEPT  first 
examines  the  known  set  of  objects  and  finds  that  none  satisfy  the  primary  conditions.  It  then 
examines  its  database  of  experimentation  techniques,  and  recalls  that  litmus  paper  changes  color 
in  the  presence  of  alcohol  and  can  be  used  to  test  for  the  hypothesized  presence  of  alcohol  vapor 
in  the  beaker.  The  litmus  paper  test  is  conducted,  via  instructions  to  the  assumed  human 
assistant  (such  as,  “make  the  litmus  paper  touch  the  air  inside  the  beaker”),  and  a  positive  result 
confirming  the  presence  of  alcohol  vapor  is  obtained. 
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ACCESS  Found  the  follovlng  applicable  to  observation  OBSERVATION- 1 
BMAP(2-C0NTAINER-LF, OSSERVATIQN-1) 

BKAP (SOILING-PROTOTYPE , OBSERVATION- 1 ) 

BMAP (SALT-DISSOLVING . OBSERVATION- 1 ) 

to* 

(8-EXPLAINS 

(SET  (PROCESS-DEFINITION  PI1  .derived  from  hoot  J!o«.  Am  vaciUvry  proem . 

(IMPLIES 

(AND  (INDIVIDUAL  tSTOVE  (CONDITIONS  (QUANTITY  (HEAT  tSTOVE))) 

(INDIVIDUAL  ALCOHOL 1  (CONDITIONS  (QUANTITY  (HEAT  ALCOHOL!)) )) 

(INDIVIDUAL  BEAKER 1  (CONDITIONS  (HEAT-PATH  BEAKER1) 

(HEAT -CONNECTION  BEAKER1  tSTOVE  ALC0H0L1) ) ) 

(HEAT- ALIGNED  BEAKER!) 

(GREATER-THAN  (A  (TEMPERATURE  tSTOVE))  (A  (TEMPERATURE  ALCOHOL1)))) 

(AND  (QUANTITY  (HEAT-FLOW-RATE  PIl)) 

(Q=  (HEAT-FLOW-RATE  PI1)  (-  (TEMPERATURE  tSTOVE)  (TEMPERATURE  ALCOHOL  1) ) ) 

(I»  (HEAT  ALCOHOL!)  (A  (HEA '-FLOW-RATE  PIl))) 

(I-  (HEAT  tSTOVE)  (A  (HEAT-FLOW-RATE  PI!)))))) 

(PROCESS-DEFINITION  PI2  /derived  from  6oWiay.  Tkt  prirntt y  proem. 

(IMPLIES 

(AND  (INDIVIDUAL  ALCOHOL  (CONDITIONS  (SUBSTANCE  ALCOHOL))) 

(INDIVIDUAL  BEAKER!  (CONDITIONS  (CAN-CONTAIN  BEAKER 1  ALCOHOL))) 

(INDIVIDUAL  ALCOHOL 1  (CONDITIONS  (CONTAINED-LIQUID  ALCOHOLl) 

(CQNTAINER-OF  ALCOHOLl  BEAKER 1) 

(SUBSTANCE-OF  ALCOHOLl  ALCHOHOL) ) ) 

(INDIVIDUAL  tSTEAMl  (CONDITIONS  (CONTAINED-GAS  t STEAM  1) 

(CONTAINER-OF  tSTEAMl  BEAKER  1) 

(SUBSTANCE-OF  tSTEAMl  ALCHOHOL))) 

(INDIVIDUAL  PIl  (CONDITIONS  (PROCESS -INSTANCE  HEAT-FLOW  PIl) 

(DESTINATION  PIl  ALCOHOL!))) 

(ACTIVE  PIl) 

(NOT  (LESS-THAN  (A  (TEMPERATURE  ALCOHOLl))  (A  (TBOIL  ALCOHOLl)))) 

(GREATER-THAN  (A  (AMOUNT-OF  ALCOHOLl))  ZERO)) 

(AND  (QUANTITY  (VAPORIZATION-RATE  PI2) ) 

(Q=  (VAPORIZATION-RATE  PI2)  (HEAT-FLOW-RATE  PIl)) 

(GREATER-THAN  (A  (VAPORIZATION-RATE  PI2))  ZERO) 

(I-  (HEAT  ALCOHOLl)  (A  (VAPORIZATION-RATE  PI2))) 

(I-  (AMOUNT-OF  ALCOHOLl)  (A  (VAPORIZATION-RATE  PI2))) 

(I*  (AMOUNT-OF  tSTEAMl))  (A  (VAPORIZATION-RATE  PI2))))) 

(PROCESS-DEFINITION  PI3  /derived  from  k**t  repttnuhmeni.  An  a nciUorp  proem. 

(IMPLIES 

(AND  (INDIVIDUAL  tSTOVE  (CONDITIONS  (HEAT-SOURCE  tSTOVE))) 

(INDIVIDUAL  PIl  (CONDITIONS  (PROCESS- INSTANCE  HEAT-FLOW  PIl)  (SOURCE  PIl  tSTOVE)) 
(ACTIVE  Pit)) 

(AND  (EQUAL-TO  (D  (HEAT  tSTOVE)  ZERO) 

(I*  (HEAT  tSTOVE  (A  (HEAT-FLOW-RATE  PIl))))))) 

OBSERVATION- 1) 

Figure  1:  Creating  an  initial  explanation  of  the  evaporation  observation.  Here,  PHINEAS  hy¬ 
pothesises,  via  behavioral  analogy,  that  the  set  of  three  boiling-related  processes  explains  the 
evaporation  observation.  At  this  stage,  these  processes  are  non-operational. 


PHINEAS  next  produces  a  Present?  query  for  the  heat  source.  In  this  case,  ADEPT  returns 
a  value  of  unknown:  unable  to  deductively  locate  such  an  object  or  experimentally  test  for  one. 
PHINEAS  responds  with 
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OUccssary?  PI1  OBSERVATION- 1} 

which  asks  if  the  ancillary  process  PU  (heat  flow)  is  necessary  to  produce  the  observation. 
ADEPT  proposes  to  repeat  the  observation,  only  this  time  thermally  isolating  the  alcohol  from 
potential  heat  flows  by  using  an  insulated  container.  When  the  amount  of  alcohol  still  decreases, 
the  need  for  an  external  supply  of  heat  is  removed  since  it  must  be  irrelevant  to  a  potential 
explanation. 

At  this  point,  the  vaporization  explanation  is  fully  instantiated  and  consists  of  a  single, 
vaporization  process.  To  make  it  fully  operational,  PHINEAS  concludes  with  the  following  two 
queries  about  its  proposed  conditioning  relations: 

(Test-Value  (A  (Temperature  alcoholl))  -i(Less-Than  (A  (TBoil  alcoholl)))) 

(Test-Value  (A  (Aaount-of  alcoholl) )  (Greater-Than  zero)  ) 

ADEPT  determines  that  the  first  condition  is  false  and  the  second  is  true.  PHINEAS  then  deletes 
the  first  condition  and  retains  the  second  condition  in  the  vaporization  model.2  At  this  stage,  the 
newly-formed  vaporization  model  is  completely  instantiated,  operational,  and  can  fully  explain 
the  observed  decrease  in  the  amount  of  alcohol  in  the  beaker. 

The  experimentally  determined  answers  to  these  theoretically-motivated  questions  enabled 
the  system  to  (1)  confirm  that  alcohol  vapor  is  present,  (2)  find  that  no  known  heat  source  exists, 
(3)  determine  that  the  type  of  heat  flow  associated  with  boiling  is  not  a  necessary  component  of 
the  newly  generated  model  of  evaporation  and  (4)  find  that  the  boiling  temperature  is  not 
required  for  the  new  process.  A  new  evaporation  model  is  conjectured  as  the  explanation.  The 
alternative  derived  from  liquid  flow  is  rejected  due  to  the  need  to  hypothesize  the  existence  of 
unobservable  entities  (e.g.,  a  destination  liquid),  which  was  not  required  of  the  evaporation  model 
(i.e.,  Occam’s  razor).3 

2.3.2.  Theory  Completion 

Once  an  operational  model  has  been  produced,  it  must  be  tested  for  consistency  with  the 
observed  behavior  by  using  it  to  envision  (Forbus,  1984)  the  possible  behaviors  of  the  current 
physical  configuration.  This  corresponds  to  asking  the  question  “what  are  the  consequences  of 
assuming  this  new  theory?”  Simulation  may  produce  unanticipated  theoretical  predictions  or 
uncover  unanticipated  interactions  with  existing  theories  (Falkenhainer,  1987).  Experimentally 
testing  theoretical  predictions  enables  the  system  to  refute  a  given  hypothesis  or  strengthen  its 
credibility.  When  unanticipated  interactions  among  theories  nullify  the  anticipated  consequences 
of  a  hypothesized  model,  these  interactions  must  be  analyzed  and  theories  revised. 

At  the  close  of  theory  completion,  a  (potentially  empty)  set  of  hypothesized  models  exists 
that  completely  and  consistently  explain  the  observed  behavior.  If  more  than  one  hypothesis  is 
produced,  the  DISCRIMINATE  task  is  used  to  eliminate  those  that  can  be  experimentally  refuted. 

1  At  model  selection  time,  the  attempt  to  restore  the  temperature  condition  fails  when  no  analogous  condition  can  be  found. 

1  Once  the  need  arose  to  hypothesue  unseen  entities,  PHINEAS's  agenda  mechanism  temporarily  abandoned  the  liquid  flow  ap- 
proach  in  favor  of  the  boiling  approach.  The  success  of  the  boiling  approach  caused  the  system  to  permanently  abandon  the  partially 
complete  liquid  flow  hypothesis.  Being  third  in  preference,  the  dissolving  hypothesis  was  never  examined. 
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2.3.3.I.  Evaporation  Example:  Completing  the  Theory  Formation  Process 
The  operational  vaporization  hypothesis  contains  the  following  three  influences: 

(I-  (Heat  alcoholl)  (A  (Vaporization-Rate  PI2))) 

(I-  (Amount-of  alcoholl)  (A  (Vaporization-Rate  PI2))) 

(1+  (Anount-of  alcohol-vaporl)  (A  (Vaporization-Rate  PI2))) 

The  influences  on  amount  are  consistent  with  the  initial  observation  that  the  amount  of  alcohol 
liquid  decreased.  However,  when  this  model  is  used  to  anticipate  what  will  happen  to  a  beaker  of 
alcohol  left  sitting  on  a  table,  it  produces  two  secondary  predictions  -  the  alcohol’s  temperature 
will  drop,  due  to  the  loss  of  latent  heat  during  vaporization,  and  the  amount  of  vapor  will 
increase.  Since  the  alcohol’s  temperature  across  time  was  not  originally  reported,  Test-Value  is 
invoked  for  the  anticipated  change  in  temperature.  ADEPT  calls  for  the  physical  scenario  to  be 
repeated  and  changes  in  alcohol  temperature  noted.  This  test  confirms  the  evaporation  theory’s 
hypothesis  and  verifies  it  as  a  complete  and  consistent  explanation  for  the  observed  phenomenon. 

2.3.3.  Theory  Revision 

Theory  revision  occurs  when  an  anomalous  observation  violates  an  existing  model.  The  first 
step  to  revising  an  imperfect  theory  is  blame  assignment,  which  is  done  in  a  layered  manner. 
First,  hypotheses  are  made  to  determine  if  the  contradiction  is  due  to  a  process  being  incorrectly 
active  or  inactive,  or  if  a  process  has  an  inappropriate  causal  effect.  Once  candidate  processes 
have  been  identified,  specific  revision  hypotheses  are  generated  to  modify  either  the  process’ 
conditions  or  its  causal  effects.  Experimentation  is  used  at  each  stage  to  eliminate  refutable 
hypotheses.  Experiments  are  proposed  until  the  correct  revision  to  the  theory  is  found  or  until 
all  hypotheses  have  been  eliminated.  This  last  case  calls  for  a  new  round  of  theory  formation. 

2.3.3.I.  Evaporation  Example:  Accounting  for  New  Anomalous  Observations 

The  newly  formed  theory  of  evaporation  predicts  that  it  will  always  be  active  as  long  as 
there  is  liquid  in  a  container.  When  this  theory  is  used  to  explain  a  second  example  of 
evaporation  -  evaporation  in  a  closed  container  -  a  contradiction  is  detected  when  the  amount  of 
liquid  stops  decreasing  after  only  a  little  liquid  has  disappeared.  Since  the  system  recognizes  this 
as  an  instance  of  evaporation  (which  stopped  too  soon),  ADEPT  is  invoked  to  revise  the  current 
model.  It  first  determines  experimentally  that  the  problem  is  due  to  a  failed  conditioning  relation 
rather  than  a  failed  effect  relation.  It  thus  queries  PHINEAS  (using  PROPQSE-REVISIOMS)  for  a  set 
of  candidate  conditioning  revisions.  PHINEAS  again  recalls  liquid  flow  and  dissolving,  due  to  their 
behavioral  similarity  and  on  the  grounds  that  they  were  both  observed  to  stop  after  being  active 
for  some  time  (i.e.,  the  relevant  problem  with  evaporation).  Through  analogous  explanations  for 
stopping,  it  proposes  two  new  candidate  quantity  conditions  -  the  amount  of  liquid  has  to  be 
greater  than  the  amount  of  vapor  (from  liquid  flow)  or  the  amount  of  vapor  has  to  be  less  than 
some  saturation  point  for  the  vapor  (from  dissolving).  .ADEPT  experimentally  refutes  the  first  (by 
constructing  an  experiment  to  increase  the  amount  of  liquid  and  observing  that  evaporation  does 
not  start  again)  and  finds  the  second  to  be  consistent  with  experiments.  This  new  saturation 
quantity  condition  is  added  to  the  evaporation  model,  yielding  a  theory  consistent  with  existing 
experiences  (Figure  2). 
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(PROCESS-3318 

Individuals  ((’V-3309  (Substance  ’V-3309)) 

(?V-3310  (Can-Con tain  ’V-3310  ’V-3309)) 

(?V-331l  (Contalntd-Llquid  ?V-3311) 

(Contaln«r-of  ?V-3311  ?V-3310) 

(Substance-of  7V-3311  7V-3309)) 

(7V-3312  (Contain»d-Cas  7V-3312) 

(Contain*r-of  7V-3312  ?V-3310) 

(Substanca-of  ?V-3312  ?v-3309))) 

QuantltyCondltions  ((Graater-Tban  (A  (Aaount-of  ?V-33il))  zero) 

(Lass-Than  (A  (Aaount-of  7V-3312))  (A  (Saturation-3326  ’V-3312) ) ) ) 
Relations  ((Quantity  (Vaporlzatlon-Rata  ?s«lf)) 

(Graatar-Tban  (A  (Vaporlzatlon-Rata  ’self))  zero)) 

Influancas  ((I-  (Haat  ?V-33U)  (A  (Vaporlzatlon-Rata  ?saif))) 

(I-  (Aaount-of  ?V-331l)  (A  (Vaporlzatlon-Rata  ?self))) 

(1+  (Amount-of  ?V-3312)  (A  (Vaporlzatlon-Rata  ’self))))) 

Figure  2:  The  hypothesised  model  of  evaporation  in  its  final,  revised  form. 


3.  An  Additional  Example:  Understanding  Osmosis 

Consider  a  case  of  forming  and  revising  a  model  for  osmosis.  In  this  example,  the  system  is 
shown  two  containers  separated  by  a  partition  which,  unknown  to  the  system,  is  semi-permeable 
(Figure  3).  It  observes  the  amount  of  solution  decreasing  in  one  container  and  increasing  in  the 
other.  The  system  is  unable  to  explain  these  observations  since  none  of  the  known  processes  are 
active.  A  number  of  processes  are  candidates  for  revision:  absorption,  boiling,  condensation, 
evaporation,  or  flow.  However,  due  to  its  ability  to  detect  behavioral  similarity,  the  system 
focuses  first  on  liquid  flow.  Using  the  Present?  query,  it  finds  two  potential  paths  for  a  flow:  the 
membrane  separating  the  two  solutions  and  the  wall  of  their  shared  container.  PHINEAS  then 
finds,  using  Test-Condition,  that  neither  object  is  Fluid-Aligned  (able  to  transport  fluids), 
since  they  are  solid  objects.  It  temporarily  removes  this  precondition  to  form  two  operational, 
competing  theories.  ADEPT,  in  response  to  a  Discriminate  task,  experimentally  rules  out  flow 
through  the  container  wall  while  all  experiments  continue  to  substantiate  the  membrane 
hypothesis.  Now  that  it  has  a  complete  and  consistent  theory,  PHINEAS  looks  for  an  analogue  to 
the  Fluid-Aligned  precondition.  After  positively  testing  for  the  effects  of  the  more  general 
aligned  concept,  PHINEAS  hypothesises  an  analogue  to  Fluid-Aligned  (called  Allgned-3417), 
which  ADEPT  determines  is  a  function  of  the  membrane  substance.  The  dependency  of  osmosis 
on  concentration  may  then  be  determined  during  revision  to  accommodate  future  observations 
(as  in  Rajamoney,  1986). 

4.  Evidence  from  the  History  of  Seience  and  Psychology 

History  is  filled  with  examples  of  analogy  being  used  for  hypothesis  generation,  followed  by 
systematic  analysis  and  experimentation  (e.g.,  Oppenheimer,  1956;  Dreistadt,  1968:  Gentner  & 
Jeziorski,  1987).  For  example,  Black’s  theories  of  latent  heats  of  vaporization  were  developed  by 
analogy  with  previously  theories  of  melting.  A  year  later,  when  a  steady  heat  source  was 
available,  he  conducted  quantitative  experiments  (Roller,  1961).  One  particularly  clear  example 
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Figure  3:  Osmosis  example.  The  solution  level  in  the  left  chamber  is  decreasing,  while  the  solu¬ 
tion  in  the  right  chamber  is  increasing. 


of  analogical  hypothesis  generation  appears  in  Carnot’s  use  of  an  elaborate  analogy  between 
water-driven  and  heat-driven  engines  in  his  development  of  the  Carnot  cycle: 

la  the  waterfall  the  motive  power  is  exactly  proportional  to  the  difference  of  level  between  the  higher  and 
lower  reservoirs.  In  the  fall  of  caloric  the  motive  power  undoubtedly  increases  with  the  difference  of  tem¬ 
perature  between  the  warm  and  the  cold  bodies;  but  we  do  not  know  whether  it  is  proportional  to  this 
difference...  It  is  a  question  which  we  propose  to  examine  hereafter.  (Carnot,  1977,  page  15) 

This  hypothesis  was  later  experimentally  verified  and  may  now  be  derived  from  the  First  law  of 
thermodynamics.  While  Carnot  assumed  the  caloric  theory,  the  Carnot  cycle  and  Carnot 
principle  laid  the  foundations  for  the  second  law  of  thermodynamics,  which  is  independent  of 
caloric  or  mechanical  theories  of  heat.  The  psychological  literature  also  includes  numerous 
studies  of  people  using  analogy  to  hypothesise  solutions  to  problems  or  explanations  of  physical 
behavior.  Construction  of  mental  or  physical  experiments  is  a  commonly  observed  post¬ 
hypothesis  behavior  (e.g.,  Clement,  1986;  Collins  &  Centner,  1987). 

5.  Related  Work 

Unlike  the  data-driven,  weak  methods  of  traditional  systems  in  machine  discovery  such  as 
BACON,  STAHL,  DALTON  (Langley  et  al,  1981,  1987)  and  ABACUS  (Falkenhainer  et  al,  1986), 
our  system  is  theory-driven  and  knowledge-intensive.  More  closely  related  to  our  integrated 
model  of  theory  formation,  revision,  and  experimentation  are  Dietterich  and  Buchanan’s  (1983) 
EG  and  Shrager’s  (1987)  IE.  EG  stressed  the  need  for  experimentation  in  theory  formation  and 
focused  on  how  experimentation  could  constrain  a  potentially  unwieldy  hypothesis  generator. 
While  we  incorporate  this  aspect,  we  also  emphasize  the  important  role  that  a  highly  focused 
theory  generator  may  have  on  experimentation.  In  IE,  the  central  claim  is  that  careful  causal 
analysis  is  unnecessary  for  tasks  such  as  forming  simple  models  of  complex  devices.  Our  primary 
concern  is  scientific  theory  formation  which  calls  for  a  more  methodical  approach,  though  it 
seems  that  our  design  could  apply  to  understanding  everyday  devices.  Our  work  also  shares  some 
similarities  with  the  IDS  system  (Nordhausen  Sc  Langley,  1987),  which  uses  experimentation  to 
induce  qualitative  models.  Unlike  IDS,  our  system  uses  analogy  to  focus  the  formation  and 
revision  of  theories,  and  directed  experiments  to  test  the  generated  and  revised  theories. 
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t.  Discussion 

Here  we  have  explored  the  strengths  of  combining  two  distinct  theory  formation  and 
revision  systems  to  form  an  integrated  system  based  on  analogy  and  experimentation.  We 
showed  how  such  an  integration  results  in  a  more  complete  account  of  the  entire  scientific 
discovery  process.  Theory  formation,  revision  and  experimentation  are  viewed  as  interrelated  and 
interdependent  processes  that  require  each  system  to  postulate  well-defined,  theoretically 
motivated  queries  to  use  the  expertise  of  the  other. 

The  evaporation  example  illustrates  many  finer  points  of  the  discovery  process.  In  the 
evaporation  example,  a  person  typically  wouldn’t  generate  all  possible  hypotheses  (e.g., 
evaporation,  liquid  flow,  absorption,  vanish,  invisible,  etc.)  and  then  discriminate  between  them. 
The  space  of  hypotheses  is  made  tractable  by  pruning  based  on  a  knowledge  of  what  can  happen 
and  a  knowledge  of  what  is  likely  to  happen  (based  on  analogical  precedents).  Also,  the  presence 
of  an  experiment  generator  enabled  us  to  test  for  the  presence  of  alcohol  vapor,  rather  than 
having  to  assume  its  existence  and  expend  effort.on  a  potentially  incorrect  hypothesis. 

We  are  still  far  from  programs  that  truly  reflect  actual  scientific  discovery  processes.  For 
example,  we  are  strongly  bounded  by  the  complexity  of  qualitative  reasoning  and  must  limit 
explorations  to  discovering  overly  simplistic  theories.  Furthermore,  the  knowledge  bases  and 
reasoning  sophistication  of  existing  discovery  systems  are  far  too  small  to  escape  the  “discovering 
Newton’s  laws  in  a  day”  phenomenon.  However,  this  work  takes  steps  towards  realizing  the  scale 
and  flavor  of  the  task.  Furthermore,  it  has  made  limitations  in  the  systems  clearer  and  provides 
a  catalyst  for  further  development.  Future  research  in  this  direction  involves  modeling  an  entire 
cycle  of  scientific  discovery  using  an  example  from  the  history  of  science  such  as  the  formation, 
revision  and,  finally,  rejection  of  the  caloric  theory  of  heat. 
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